-
Notifications
You must be signed in to change notification settings - Fork 21.4k
cmd/evm: add stdin support to blocktest command #32824
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Enable blocktest to read filenames from stdin when no path argument is provided, matching the existing statetest behavior. This allows efficient batch processing of blockchain tests. Usage: - Single file: evm blocktest <path> - Batch mode: find tests/ -name "*.json" | evm blocktest 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
The lint is red @bshastry |
9d2a754
to
0e8268a
Compare
0e8268a
to
94b4de0
Compare
Been looking a bit into this. Its seems the whole concept of EndMarker should be removed. What I guess you want is to write out the results between the traces not only at the end of the test, right? This should be possible with the traceResult struct. I will push a commit on top which changes this, we can remove it again if you don't agree |
Yeah, basically I want a read protocol for jsonl that signals when blocktest results are complete, so I can break the jsonl scanner loop. trace line 1 trace line 2... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This commit addresses all feedback from @rjl493456442: 1. Result printing location: Removed duplicate report() call from inside test loop. Now uses dedicated traceEndMarker on stderr instead of printing full results mid-execution. 2. Fork assignment: Fork now always assigned regardless of tracer or pass/fail status. Root only assigned when test succeeds, following correct semantics. 3. Log suppression: Removed automatic log suppression. Users can control logging via standard --verbosity and --log.file flags. Only one rare INFO log exists in blocktest path. The traceEndMarker provides clear delimiter for trace parsers (e.g., goevmlab) without duplicating test results. Format: {"testEnd":{"name":"...","pass":true,"fork":"...","root":"..."}} This follows precedent from Nethermind PR ethereum#9432. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔍 Code Review Summary
The review focused on adding stdin support to the evm blocktest
command, aligning its functionality with statetest
for improved batch processing. The implementation is straightforward and achieves its goal. The primary risk identified is code duplication and a minor behavioral inconsistency, for which a clear refactoring suggestion is provided to create a unified file collection utility. Overall, the change is a valuable, low-risk enhancement to the developer toolkit.
📋 Reviewed Changes
- cmd/evm/blockrunner.go: Modified to read test file paths from stdin when no command-line arguments are provided.
- cmd/evm/main.go: Contains existing utility functions like
collectFiles
which are related to the new logic. - tests/block_test_util.go: Test utilities, reviewed for context and potential impact from the changes.
📊 Architecture & Flow Diagrams
Diagram 1
graph TD
A["evm blocktest"] --> B{"Args provided"}
B -->|Yes| C["Read paths from args"]
B -->|No| D["Read paths from stdin"]
C --> E["Expand directories collect files"]
D --> E
E --> F["Run tests on file paths"]
🤔 Review Quality Assessment
The review demonstrates comprehensive coverage, correctly identifying code duplication and the opportunity for a shared utility. The synthesis process refined this insight into a more holistic and actionable recommendation that includes refactoring statetest
for consistency and addresses subtle behavioral differences. Confidence in the final, consolidated suggestions is high, as they significantly improve the long-term maintainability and robustness of the CLI tool.
💡 Suggestions Summary
- Inline Comments: 0 suggestions on modified code lines
- General Suggestions: 2 suggestions for overall improvements
📝 Additional Suggestions
The following suggestions apply to code outside the current PR diff:
1. 🔴 To improve maintainability and ensure consistent behavior across commands, refactor the file path collection logic into a shared utility function in cmd/evm/main.go
. This new function, collectPathsFromInput
, should handle reading from both command-line arguments and stdin, and also manage directory expansion for all inputs. This eliminates code duplication between blocktest
and statetest
and ensures consistent behavior.
File: cmd/evm/main.go
(Line 360)
Confidence: 95%
// collectPathsFromInput reads paths from command-line arguments or stdin and expands them.
func collectPathsFromInput(ctx *cli.Context) []string {
var paths []string
if ctx.NArg() == 0 {
// Read from stdin if no arguments are provided.
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
paths = append(paths, scanner.Text())
}
} else {
// Read from command-line arguments.
paths = ctx.Args().Slice()
}
// Expand all paths, whether from stdin or args, to discover test files.
var expandedPaths []string
for _, path := range paths {
expandedPaths = append(expandedPaths, collectFiles(path)...)
}
return expandedPaths
}
2. 🔴 Update blockTestCmd
to use the new collectPathsFromInput
helper function. This simplifies the command's implementation, removes the duplicated logic, and ensures it benefits from the centralized, consistent method for discovering test files.
File: cmd/evm/blockrunner.go
(Line 158)
Confidence: 95%
paths := collectPathsFromInput(ctx)
if len(paths) == 0 {
return errors.New("no test files found")
}
Enable blocktest to read filenames from stdin when no path argument is provided, matching the existing statetest behavior. This allows efficient batch processing of blockchain tests.
Usage:
🤖 Generated with Claude Code